Overlapped Speech Detection and speaker counting using distant microphone arrays

We study the problem of detecting and counting simultaneous, overlapping speakers in a multichannel, distant-microphone scenario. Focusing on supervised learning approach, we treat Voice Activity Detection (VAD), Overlapped Speech (OSD), joint VAD OSD (VAD+OSD) speaker unified way, as instances general Counting (OSDC) multi-class problem. consider Temporal Convolutional Network (TCN) Transformer based architecture for this task, compare them with previously proposed state-of-the art methods Recurrent Neural Networks (RNN) or hybrid Convolutional-Recurrent (CRNN). In addition, propose ways exploiting multichannel input by means early late fusion single-channel features spatial extracted from one more microphone pairs. conduct an extensive experimental evaluation AMI CHiME-6 datasets purposely made synthetic dataset. show that Transformer-based performs best among all architectures neural network localization outperform signal-based significantly improve performance compared to only. Finally, find training objective improves VAD+OSD objective.